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Figure 1. Left: A device’s magnetometer (blue arrow) points towards the earth’s magnetic field (North). Center: A small non-obtrusive magnet is 
attached to the user’s index finger with the South Pole up. When this finger touches the screen, the magnetometer reports the direction and strength 
of the magnet’s field and which allows for identifying the index finger. The magnet’s orientation can be used to define different fingerprints as to 
dinstinguish different fingers. Right: The middle finger is equipped with a magnet with the North Pole up. 


ABSTRACT 

We present magnetic fingerprints; an input technique for mo¬ 
bile touchscreen devices that uses a small magnet attached to 
a user’s fingernail in order to differentiate between a normal 
touch and a magnetic touch. The polarity of the magnet can 
be used to create different magnetic fingerprints where this 
technique takes advantage of the rich vocabulary offered by 
the use of multitouch input. User studies investigate the accu¬ 
racy of magnetic fingerprint recognition in relation to magnet 
size, number of magnetic fingerprints used; and size of the 
touchscreen. Studies found our technique to be limited to us¬ 
ing up to two fingerprints non-simultaneously, while achiev¬ 
ing a high classification accuracy (95%) but it nearly triples 
the number of distinguishable multi touch events. Potential 
useful applications of this technique are presented. 
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INTRODUCTION 

Touch-screens have become a de facto standard of input for 
mobile devices as they most optimally use the limited input 
and output space that is imposed by their form factor. A single 
touch event is typically interpreted using spatial information, 
but precisely manipulating content is often challenging due 
to small screens, occlusion, variation between users, and fin¬ 
ger orientation [13]; which requires virtual buttons on touch¬ 
screens to be larger than the physical buttons they replace. 


To increase the vocabulary of touch interaction, a number of 
techniques build on or enhance single touch events. Single 
touch events can be combined into multi-gestures, such as a 
pinch, but these do not scale up very well due to technical 
and physiological constraints and do not offer the same ac¬ 
curacy as a single touch input. Temporal features of single 
touch events can be taken into account to define unique ges¬ 
tures, such as multi-taps or dwelling. Different types of touch 
events can be defined based on the size of a touch event [5]. 
Temporal features also can be combined with spatial informa¬ 
tion, e.g., so called micro-gestures [22], but this type of input 
is considered less efficient and more error prone than when 
using single touch [11]. 

Beyond counting and timing, a significantly larger touch vo¬ 
cabulary could be created if we could distinguish between 
different types of single touch events. Some recent work has 
already explored this idea by creating distinct touch events 
through the incorporation of additional sensing information 
acquired with an accelerometer [12] or microphone [10] (see 
Related work following) A limitation of these techniques is 
that they cannot be used as part of multi-touch input, where 
the largest increase in touch vocabulary may be achieved. 

Though magnets have been previously explored as a mobile 
input technique those only involve around-the-device interac¬ 
tion [9, 18] or input with a magnet embedded in an object, 
such as a stylus [20]. Our approach is novel as we instrument 
a finger with a small magnet, which allows us to distinguish 
magnetic fingers from non-magnetic fingers. Our approach 
requires the smallest amount of user instrumentation. Unlike 
existing approaches, out technique can be used to augment 
multitouch input and therefore significantly increase current 
touch vocabulary. 
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RELATED WORK 

A number of input technologies are related to our approach. 
We differentiate approaches that augment single touch input 
from those that explore using magnets for input. 

Augmenting single touch input: The spatial and temporal 
features of a touch event can be used to define micro-gestures. 
Microrolls 122], e.g., unidirectional or circular rolls or rubs 
made with the thumb can be used to define new types of in¬ 
put, such as opening up a contextual menu. Sixteen differ¬ 
ent micro gestures can be defined with an overall recognition 
accuracy of 95%, though this requires per-user calibration. 
Thumbrock 14] improves upon the aforementioned approach 
and achieves a 96% accuracy in recognizing thumb rolls with¬ 
out calibration. SimPress 12] uses the contact area of a touch 
event generated by the index finger as a form of simulated 
pressure to generate different types of single touch events. 
Fat thumb 15] extends this approach to the thumb which has 
a larger surface area. This approach lends itself well for sin¬ 
gle hand mobile input where pressing the thumb more firmly 
to the touchscreen can be used for zooming in and pressing 
softly for zooming out. The total number of contact size lev¬ 
els that can be feasibly used or recognized is not investigated. 
A benefit of these approaches is that they do not require any 
form of instrumentation though it does require per user cal¬ 
ibration. Though these approaches work on existing touch¬ 
screen devices, a limitation of combining spatial and tempo¬ 
ral information of touch events is that recognition is slower 
and more error-prone 111] than using single touch events. 

Hinckley and Song 112] present a technique that combines 
capacitive touch sensing with acceleration data, which allows 
for distinguishing “hard” from “soft” taps and swipes. This 
approach offers a natural semantics as a hard tap can be used 
to drill down into a menu and a soft tap to go down a sin¬ 
gle level. No results are presented on the accuracy of this 
approach, though qualitative results are reported. A similar 
approach is Gripsense 17] which uses inertial sensors and a 
vibrator to measure pressure on a touchscreen. When the 
user touches the screen the vibrator is briefly activated and 
the damping of vibrations is measured using the accelerom¬ 
eter and gyroscope. This information can then be used to 
distinguish three levels of finger pressure with high accuracy 
(95%) and distinguish different hand postures, e.g., a thumb 
from an index finger with 84% accuracy. Both approaches 
work on current mobile touchscreen devices without any ex¬ 
tra sensors. 

Tapsense 110] combines capacitive touch sensing with acous¬ 
tic information acquired with an external high fidelity stetho¬ 
scope. Capacitive touches made by different objects or parts 
of the finger make different sounds, which can be classified 
accordingly. For single-handed mobile finger input, four dif¬ 
ferent types of touches made by the tip, pad, nail and knuckle 
can be recognized with a high 95% accuracy, but it requires 
per user calibration. This technique supports multiuser in¬ 
put, though touch events with different objects cannot over¬ 
lap. Fiberio 114] a rear projected multitouch table that can 
optically sense a user’s fingerprints. This technique relies on 
a novel fiber optic plate and unlike magnetic fingerprints, it 


cannot be used on existing mobile touchscreen devices. Due 
to the rear projection requirement, this technique is confined 
to interactive tables. 

A limitation of the aforementioned approaches is that it is 
challenging to use them as part of a multi-touch gesture made 
with a single hand. Pressure based approaches only seem to 
work with a single finger 17,12]. A thumb roll 122] or a knock 
with a knuckle 110] are impractical to use as part of a multi- 
touch gesture, though this is an area in which the largest in¬ 
crease in touch vocabulary may be achieved. 

Magnet based input: A magnetometer measures the earth’s 
magnetic field, and -when combined with an accelerometer- 
can be used to determine the absolute 3D orientation of a mo¬ 
bile device. Magnetometers are typically used as a compass 
for navigation or to point out contextual geographic informa¬ 
tion and are thus widely available in current mobile touch¬ 
screen devices. A magnetometer also senses the strength and 
direction of the magnetic field of a magnet. Because mag¬ 
nets are cheap and do not require an external power source 
a number of approaches have been explored that use mag¬ 
nets for wireless around-the-device input. Abracadabra 19], 
Nenya 11] and MagiTact 118] use magnet rings to allow for 
wireless analog input on mobile devices. Abracadabra and 
Magitact use the magnet’s distance from the magnetometer to 
control a cursor or scroll through a list. By rotating the ring, 
Nenya uses changes in the direction of the magnetic field to 
scroll through a list. MagiWrite 117] enables text input by 
allowing its users to write digits in the air using a handheld 
magnet. 

A different set of input techniques embed the magnet into 
an object. GaussSense 120] embeds a magnet into a stylus, 
which then facilitates advanced stylus interaction on touch¬ 
screens where the orientation and tip pressure of a stylus 
can be accurately sensed and used for input. GaussSense 
requires an external sensor grid that is attached to the back 
of the mobile device. User studies investigate the accuracy 
with which finger touches can be distinguished from stylus 
touches. Magpen 115] is a similar technique that uses a mag¬ 
netic stylus but works with a device’s internal magnetometer. 
A set of rotary gestures with the pen are proposed. 

Appcessories 13] presents a number of tangible objects with 
magnets embedded in these object. A number of novel in¬ 
teraction options are presented, e.g., objects placed on the 
touchscreen can be identified using the direction of the mag¬ 
net’s magnetic field. When combined with their spatial loca¬ 
tion, this information can be used to activate different com¬ 
mands. By rotating an object the change in the direction of 
the projected magnetic field is interpreted to allow for pro¬ 
viding analog forms of input, such as scrolling through a list. 
GaussBits 119] is a similar approach that uses objects with 
magnets embedded in it. GaussBits uses an external mag¬ 
netic sensor grid and therefore offers a much higher accuracy 
than when using a device’s internal magnetometer. GaussBits 
supports wireless 3D interaction. MagGetz 116] embeds mag¬ 
nets in physical controllers like switches, buttons, sliders and 
joysticks and changes in the magnetic field are interpreted by 
a device’s magnetometer. 
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Figure 2. A single magnetic fingerprint doubles the number of distinguishable single and multi-touch events, i.e., we can distinguish two 
unique single touch events and two different pinch gestures. 


DESIGN OF MAGNETIC FINGERPRINTS 

We developed a novel input technique, called magnetic fin¬ 
gerprints, that is based on the following observations. For 
input, existing magnet input techniques have either explored: 

• Around-the-device interaction where the user either wears 
the magnet as a ring [1, 9,18] or holds a magnet in their 
hand [17]. 

• On-screen interaction either with magnet embedded in an 
object [3,19] or stylus [15,16,20,21]. 

Rather than using a magnet as a standalone input technique, 
we use a magnet to augment finger based touch input. A sig¬ 
nificant benefit of this approach is that magnetic fingerprints 
can be used as part of a multitouch gesture, such as a pinch 
(see Figure 2). Another benefit is that it doesn’t require the 
user to learn new gestures. Though current mobile touch¬ 
screen devices typically rely on capacitive sensing, magnetic 
fingerprints could in theory work with any type of touch¬ 
screen technology (resistive/optical) as long as it features a 
magnetometer, which excludes interactive table tabletops. 

A permanent magnet can provide two types of input: (1) the 
sensed strength of the magnet depends on its distance to the 
magnetometer and this property can be used to facilitate ana¬ 
log forms of input, such as scrolling [9,17]; or (2) the direc¬ 
tion of its magnetic field can be interpreted to provide dis¬ 
crete forms of input, i.e., selecting an item [3, 9]. A mag¬ 
netometer reports a 3D vector v, which -when there is no 
magnetic interference- points towards North n (see Figure 
l:left). When a magnet approaches the screen, the vector re¬ 
ported by the magnetometer will change to the value of the 
magnetic field projected by the magnet rh. The direction and 
magnitude of m differs significantly from the earth’s mag¬ 
netic field, i.e., fh ^ n and \m\ ^ |n| (for strong enough 
magnets). The strength of the magnet or the orientation of 
its projected field will generate unique changes in the magne¬ 
tometer, i.e., c = n + m when the magnet is moved towards 
the magnetometer. 

It is this property that our approach exploits by instrument¬ 
ing a fingernail with a small magnet, where the strength of 
the magnet or the direction of the magnetic field can be used 
to define unique “magnetic fingerprints” that can be distin¬ 
guished through unique values of c. Various novel input tech¬ 
niques [6,24] have explored instrumenting the user’s finger 
with sensors or even a non-obtrusive magnet [23]. Different 
approaches could be used for attaching a magnet to a user’s 
finger, such as with a rubber band [23], though it seems most 
practical for the user to wear a capacitive thimble with a mag¬ 


net embedded in it. Our preliminary experiments showed that 
wearing a magnetic ring [9,18] does not allow for accurately 
distinguishing individual fingers. A capacitive glove could 
accommodate multiple magnets and is available at low cost. 

A major consideration in the definition of magnetic finger¬ 
prints is its form factor; e.g., the size and the shape of a mag¬ 
net that needs to be attached to the user’s fingernail. A mag¬ 
net’s strength depends -among other factors- on its size. A 
larger magnet is more easily detected, especially when the 
magnet is held further away from the magnetometer. For 
practical and aesthetic reasons it is more desirable to use the 
smallest possible size magnet. A challenge with using mag¬ 
net strength is that different sized magnets may generate the 
same value for c depending on the distance of the magnet 
from the magnetometer. To accurately discriminate differ¬ 
ent sized magnets, we need to incorporate where the finger 
touches the screen and a classifier needs to be trained using 
measured values of c for different magnet sizes and locations. 

If we define magnetic fingerprints based on the direction of 
the magnetic field, the implementation is significantly simpler 
as we do not need to collect training data and a higher accu¬ 
racy may be achieved as we only need to look at the direction 
of change of the components of c. Because this approach 
doesn’t require incorporating where the finger touches the 
screen, touch less input is one possible application of mag¬ 
netic fingerprints (see Example Applications). A previous 
study found that discerning magnets based on their strength is 
very difficult unless they are really different in size [3], which 
leads to form factor issues. Given the simpler implementa¬ 
tion and possible higher accuracy, we only explore defining 
fingerprints using the direction of the magnetic field. 

Because we combine touch sensing with sensing the presence 
of a magnetic fingerprint, we only require n — 1 magnetic fin¬ 
gerprints to distinguish n different fingers, as sensing the ab¬ 
sence of a magnetic fingerprint in a touch event also defines a 
unique input event. Mobile touchscreens can detect up to five 
different types of touch gestures, with single touch, two and 
three finger multitouch gestures being most practical. A mag¬ 
netic fingerprint may be chorded with other non-magnetic 
fingers to form multi-gestures. Using a single magnetic fin¬ 
gerprint, nine unique gestures (two single touch, and seven 
multitouch gestures) can be defined (see Figure 2). With two 
magnetic fingerprints, this number theoretically increases to 
16 unique gestures, more than three times the amount of what 
current touchscreen devices are able to detect. Besides prac¬ 
tical issues, e.g., fingers could get stuck to each other when 


3 



Figure 3. Left: for our studies a finger is instrumented with a small 
neodymium magnet (7.9mm diam.) using an adhesive. Right: alterna¬ 
tively the user wears a capacitive thimble with an embedded magnet. 

magnets attract each other, another potential problem with 
chording magnetic fingerprints could be magnetic interfer¬ 
ence. Current magnetometers only report the sum of the com¬ 
plex magnetic field between multiple magnets, which makes 
it difficult to recognize this gesture. User studies provide in¬ 
sight into how many fingerprints can be used and whether 
chording is feasible. 

STUDY 1: FORM FACTOR AND SCALABILITY 

For this study, we are interested in understanding how the ac¬ 
curacy of recognizing magnetic fingerprints varies depending 
on magnet strength and the number of magnetic fingerprints 
used. A larger magnet is detected more easily but magnetic 
interference may increase when using multiple magnets. 

Instrumentation 

An Apple iPhone 4, with a 1.94” x 2.91” (640 x 960 pixel) 
display was used for our experiment. For this study, we limit 
ourselves to touchscreen input on smartphones, as an increase 
in input space offers the largest benefit for this specific plat¬ 
form due to their limited screen real estate. When held in 
the portrait position, the iPhone’s magnetometer is located 
in the upper right comer. We used N40 grade, disc-shaped 
Neodymium magnets, which is the strongest type of commer¬ 
cially available permanent magnet at a low cost (< $0.10). A 
disc-shaped magnet was chosen as it is least obtrusive and be¬ 
cause this shape has been successfully used in a related haptic 
output technique [23]. This shape is more easily attached to a 
nail than, for example, a cube shaped magnet. Disc magnets 
are axially magnetized with poles located on the opposing 
fiat circular surfaces of the disc. Preliminary trials confirmed 
that the classification accuracy of using magnet strength for 
defining magnetic fingerprints was much lower than when us¬ 
ing the direction of the magnetic field. When using multiple 
magnets, they need to differ in size significantly as to accu¬ 
rately classify them, which is detrimental to the form factor 
of this technique. When using the direction of the magnetic 
field, a magnet of the same size can be used. For these reasons 
our experiment is limited to evaluating up to two magnetic 
fingerprints that are defined by the direction of the magnetic 
field (each pole can only be worn face up). As mobile input 
is typically limited to single hand input, we deem a poten¬ 
tially three-fold increase in touch vocabulary by exploring the 
use of two magnetic fingerprints large enough for our study. 
Based on preliminary trials, we decided to explore the follow¬ 
ing diameters for our magnets (12.7, 7.9 and 3.2 mm with a 
magnet height of 0.8mm). We marked the north pole of the 
magnet with an X and attached the magnet to the subject’s 
finger nail using a reusable dot adhesive (see Figure 3deft). 



Figure 4. Left: the 4x3 grid used for single target selection task. Right: 
colored targets that are rendered for the multi-touch selection task. 


Participants 

We recruited 10 participants (3 female, average age 27.5, 
SD = 3.5). All subjects were right handed and none had 
any self-reported non-correctible impairments in perception 
or impairments in motor control. All subjects were familiar 
with touchscreen input, as they all owned a smartphone. 

Procedure 

The strength of the magnet’s magnetic field sensed by the 
magnetometer varies depending on the distance between the 
magnet and the magnetometer. To understand what size mag¬ 
net is required to accurately detect the magnetic fingerprint at 
every location on the screen, we had subjects perform a single 
target selection task. An iOS application was written that re¬ 
quires subjects to perform a target selection task on a 4x3 grid 
(see Figure 4). A target is rendered using an orange circle that 
measured 220 by 200 pixels (see Figure 4 left.) The (X,Y, Z) 
values of the magnetometer were recorded in a log file with a 
sample rate of 40Hz. We record the time and location (X,Y) 
of each touch event. 

We used a repeated measures within-subjects design. Inde¬ 
pendent variables were magnet size and number of magnetic 
fingerprints. For one magnetic fingerprint, we instrumented 
the middle finger with a magnet (North pole up) and used 
the middle and index finger for target selection. This setup 
was used as the index finger is most frequently used for mo¬ 
bile input. For two magnetic fingerprints, we instrumented 
the index (North pole up) and ring finger (South pole up) 
and used these fingers and the middle finger for the target 
selection task. This setup was chosen to minimize magnetic 
interference by having the largest possible distance between 
them. An additional constraint that we identified using a pre¬ 
liminary trial is that better results were achieved for when the 
user retracts a magnetic finger (place it in the palm) when this 
finger is not involved in the target selection, as this minimizes 
false recognitions. Though these specific conditions optimize 
the use of our technique, it helps establish an upper bound 
on the best performance. We counter-balanced the conditions 
among subjects. The phone was placed on a fiat surface in 
the landscape orientation and subjects used their dominant 
hand for target selection. Each finger selected 24 targets with 
two targets per cell in randomized order. Subjects did not 
switch between fingers, as we were primarily interested in 
understanding how the accuracy varies depending on where 
the screen is touched. 

Results 

To determine the accuracy with which magnetic fingerprints 
can be distinguished from each other and from non-magnetic 
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Table 1. Classifier accuracy (standard dev) 


magnet 

diameter (mm) 

1 # magnetic fingerprints 

1 

2 

12.7 

98.47 (1.23) 

95.71 (1.68) 

7.9 

99.02 (0.95) 

94.88 (2.73) 

3.2 

92.18 (3.49) 

80.74 (3.68) 


touches, we trained a classifier, i.e., a support vector machine 
(SVM) using the LibSVM library. To avoid over-fitting our 
SVM to the training data, we used ten-fold cross-validation 
using all our data for a total of 480 data points for one mag¬ 
netic fingerprint and 720 data points for two fingerprints. The 
data was labeled and partitioned into ten equal size subsam¬ 
ples. A single subsample is retained for testing where the 
other nine subsamples are used as training data. The cross- 
validation process is repeated 10 times with each of the sub¬ 
samples used as the validation data, and a single estimate 
is achieved by averaging the results from each fold. Vari¬ 
ous features were explored to train the SVM, with the best 
performance achieved for data points containing 62 features, 
which included: (1) touch location (X,Y); and (2) differential 
magnetometer data (0.5 seconds) preceding the touch event, 
which included 20 vectors (X,Y, Z) of c with c = m — Vt 
where m is measured at the touch event t and Vt in 2bms 
decrements. Parameters were determined experimentally to 
yield the best results with no filtering applied. Differential 
magnetometer values are used as raw magnetometer values 
vary depending on how the phone is oriented with regard to 
the earth’s magnetic field, which could vary between trials. 

Table 1 lists the classification accuracy for each condition - 
including their standard deviation- based on the classification 
accuracy of each fold. We performed a two-way ANOVA 
and found a significant interaction for size and number of 
magnetic fingerprints regarding classifier accuracy (^ 2,54 = 
16.926, p = .00, partial r]‘^ = .385). Post-hoc analysis re¬ 
vealed that 3.2 mm magnet differs in classifier accuracy from 
the other sizes (p = . 00 ) with no difference between the 
7.9mm and 12.7mm magnets (p = 0.99). A single magnet 
yields the best performance (p = . 00 ) with an accuracy of 
99.02% (SD=.95) for the 7.9 mm magnet. All errors were 
due to not correctly recognizing the magnetic fingerprint with 
71% of the errors made by a single subject. For two magnets, 
the best performance is 95.71% (SD=1.68) using the 12.7mm 
magnet, with errors uniformly distributed among subjects and 
types of fingers. Errors were more likely to occur in cells far¬ 
thest away from the magnetometer. 

STUDY 2: MULTI-TOUCH GESTURE RECOGNITION 

Where our first study focused on single finger target selec¬ 
tion, our second study evaluates the classification accuracy 
of multi-touch gestures. We focus on two-finger gestures and 
require subjects to switch between different fingers, which re¬ 
sembles a more practical usage of our technique that we did 
not evaluate in our first study. Using a preliminary experi¬ 
ment, we analyzed whether it was feasible to chord two mag¬ 
netic fingerprints. The thumb and index finger were instru¬ 
mented with the 12.7mm magnet, which yielded the best per¬ 
formance in our first study. Two fingerprints allow for defin¬ 


ing three unique pinch gestures (thumb-index, thumb-middle, 
index-middle). Three subjects performed a number of differ¬ 
ent pinches where they had to pinch two targets together. We 
used fixed targets that were defined around the center of the 
screen. An SVM was trained using the same features as our 
first study, which yielded an overall classification accuracy 
of 74.44% (SD=12.88) with an accuracy of 60.00% for the 
pinch gesture involving both magnetic fingerprints. Based on 
this result, chording magnetic fingerprints does not seem fea¬ 
sible and we limit our study to multi-gestures that use a single 
magnetic fingerprint at a time. 

Instrumentation 

Based on previous results, we used the magnet with a 7.9 mm 
diameter and we used the same iPhone 4 for our experiment. 

Participants 

We recruited eight people (1 female, average age 25.8, 
SD=4.7) with four people having participated in our first 
study. All subjects were right handed, owned a touchscreen 
device and none reported any non-correctable impairment. 

Procedure 

We used the same application that we developed for our first 
experiment, but we modified it so that two targets are ren¬ 
dered that need to be moved together using a pinch gesture 
(see Figure 4:right). For the same reason as our first study, we 
leave the index finger free and instrument the middle finger 
with a magnet to allow for defining two unique pinch gestures 
(see Figure 2). What pinch gesture to provide is indicated us¬ 
ing visual cues, e.g., a blue target indicates using a magnetic 
fingerprint where an orange target a non-magnetized finger. 
Ten unique combinations of targets were defined with one cell 
distance between the targets. These combinations consisted 
of six horizontal pinches and four diagonal pinches. To es¬ 
tablish an upper bound on the performance of this setup, sub¬ 
jects were asked to retract their magnetic finger when it was 
not involved in the target selection to minimize false recog¬ 
nitions. Subjects used their dominant hand for gesture input. 
For each type of pinch, subjects selected 20 targets for a total 
of 40 targets with the order of targets and type of gesture be¬ 
ing randomized. We also record the number of attempts made 
for each correct pinch gesture. 

Results 

We employed an SVM using ten-fold cross-validation for a 
total of 160 data points per type of pinch gesture. Using 
64 features, e.g., 4 for the touch locations and 60 for dif¬ 
ferential magnetometer data, we achieved an overall classi¬ 
fication accuracy of 97.19% (SD=3.11) with an accuracy of 
96.88% (SD=4.15) for the non-magnetic pinch and 97.50% 
(SD=3.23) for the magnetic pinch, with no significant dif¬ 
ference between them (fig = .172, p = .693). An anal¬ 
ysis of errors found a relatively uniform distribution of er¬ 
rors among subjects, with slightly higher error rate for the 
cell farthest away from the magnetometer. There was no 
significant difference in classification accuracy with our first 
study (f 18 = .401, p = .093), which indicates that switching 
between fingers for input does not negatively affect perfor¬ 
mance. Users required an average of 1.12 attempts (SD=.18) 
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Table 2. Classifier accuracy (standard dev) 


magnet 
height (mm) 

grid size 

8x6 

6 y 2 X 5 

4x3 

0.8 

79.40 (4.99) 

81.98 (5.50) 

93.68 (5.45) 

1.6 

81.64 (4.76) 

81.33 (2.77) 

93.75 (5.25) 

2.4 

86.85 (3.70) 

88.11 (4.08) 

98.43 (2.59) 


for each type of pinch with no significant difference in num¬ 
ber of attempts between both pinches (ty = .00, p = 1.00). 
Besides demonstrating the feasibility of using a single mag¬ 
netic fingerprint to augment multitouch input we found that 
using the middle finger for a pinch gesture is just as effective 
as using the index finger. 

STUDY 3: TOUCHSCREEN SIZE 

Tablets have become a popular mobile computing platform 
but they typically feature larger touchscreens than smart¬ 
phones. For this study, we evaluate whether magnetic fin¬ 
gerprints can be used on tablets. Due to their larger screen 
size, stronger and larger magnets need to be used. We limit 
our experiment to evaluating a single magnetic fingerprint. 

Instrumentation 

For this experiment, we used the popular Apple iPad 4 tablet, 
which features a 9.50” x 7.31” (2,048 x 1,536 pixel) display 
(9.7” diagonal). We were unable to verify whether the iPad 
features the same magnetometer as the iPhone. We used the 
largest diameter magnet from our first experiment (12.7 mm). 
As the 12.7 mm magnet is already at the size of a nail, we 
explore magnets with larger height. Based on experiments, 
we evaluated the following heights (0.8, 1.6 and 2.4 mm). 

Participants 

We recruited eight subjects (1 female, average age 27.0, 
SD=4.6). Four subjects had participated in prior studies. All 
subjects were right handed, owned a smartphone and none 
reported any non-correctable impairment. 

Procedure 

Subjects performed the same target selection task as in our 
first study. Instead of using a 4x3 grid, we use an 8x6 grid 
to allow for a comparison of results using a similar cell size 
(accommodated for differences in screen resolution). Though 
the iPhone 4 and iPad 4 have slightly different aspect ratios, 
we did not deem this to be a significant problem. For a sin¬ 
gle magnetic fingerprint, we instrumented the middle finger 
with a magnet and used the middle and index finger for target 
selection. Each finger selects 48 targets in the grid in ran¬ 
domized order for a total of 96 targets per magnet size. We 
counter-balanced the conditions among subjects. 

Results 

An SVM was trained using the same features as for our first 
study for a total of 384 data points per type of finger. To un¬ 
derstand the effect that touchscreen size has on classification 
accuracy, we trained SVMs using subsets of our grid. We 
used a grid of 6V2 x 5 (iPad mini) and 4x3 (iPhone 4) to allow 
for a comparison with results from the first study. Table 2 lists 


the results. For the 8x6 grid, a one-way ANOVA found a sta¬ 
tistically significant difference in accuracy between magnets 
(^4,7 = 7.108, p < .05) where a post-hoc analysis showed 
that the 2.4 mm magnet outperforms the 1.6 mm (p = .049) 
and the .8 mm (p = .00) magnets. An accuracy of 86.85% 
(SD=3.70) is achieved for the 9.7” iPad with a slightly higher 
accuracy (88.11%) for 8” tablets (iPad Mini). Errors were 
uniformly distributed among subjects and among types of fin¬ 
gers. Similar to our prior results, errors were more likely to 
occur in cells farthest away from the magnetometer; however, 
we also observed false recognitions in the cells closest to the 
magnetometer. Though we ensured subjects retracted their 
magnetized finger, it seems due to the larger screen and the 
size of the magnet used, false recognitions become difficult 
to avoid. This also explains the lower accuracy found for the 
.8 mm magnet for the 4x3 grid. 

EXAMPLE APPLICATIONS 

We illustrate the usefulness of magnetic fingerprints using a 
number of example applications. 


Mode Switching 



Figure 5. Left: regular characters are provided with the index finger and 
special characters with a magnetic fingerprint. Middle: Before touch¬ 
ing the screen, we can sense the presence of a magnetic fingerprint and 
switch to the numerical keyboard. Right: a special character is selected. 

Providing special characters on mobile virtual keyboards of¬ 
ten requires switching to a special keyboard, which is ineffi¬ 
cient as it requires multiple user inputs to select the right key. 
To address this problem, Harrison et al. presents a solution 
using their Tapsense technique [9] in which special characters 
can be provided using different parts of the finger (pad,nail). 
This approach integrates multiple keyboards into a single key¬ 
board and renders multiple characters on each key, which may 
cause keys to become illegible. Magnetic fingerprints can be 
used for efficient mode switching. For a mobile keyboard, 
users type regular characters with their index finger and use a 
magnetic fingerprint on their middle finger to provide a spe¬ 
cial character. What is novel about this approach is that we 
can already sense when a magnetized finger is approaching 
before it touches the screen and switch to the corresponding 
keyboard. This allows users to hover with their magnetized 
finger over the keys to find the special character before select¬ 
ing it. Our approach could support three different keyboards 
while allowing for legible characters on keys. 

Rotary Gestures 

Knobs or sliders are GUI elements that can be used for ad¬ 
justing continuous input values, such as volume or pitch in a 
music application. It is more desirable to use a knob for mo¬ 
bile touchscreen input, as they require less screen real estate 
than a slider [8]. Knobs are difficult to manipulate using touch 
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Figure 6. Left: a knob is touched with a magnetic fingerprint. Right: 
When the finger rotates, changes in the magnetic field adjust the knob. 


as this cannot be achieved with a single touch event and re¬ 
quires interpreting a two finger rotate. Two finger rotates are 
difficult to perform with a single hand and requires increasing 
the size of the knob to allow for bimanual access. Bianchi et 
al. [3] demonstrates providing rotary input with a magnet em¬ 
bedded in an object. The object is placed on the touchscreen 
and when it is rotated changes in the direction of its magnetic 
field are interpreted to adjust a value. A magnet attached to 
a user’s finger can be interpreted in the same way and allows 
users for to precisely manipulate a knob by rotating their fin¬ 
ger instead of an object or using two fingers. Experiments 
show a feasible input range of nearly 180°. This gesture could 
also exploit the metaphor or screw/unscrew gestures where a 
value, e.g., a slider bar handle, can be fixed temporarily. 

Multi^-Touch Gestures 



Figure 7. Left: a non-magnetized pinch scales a photo. Right: a pinch-in 
with a magnetic fingerprint picks up the photo where it can be dragged 
to a new location and a pinch-out drops the photo. 

Multi-touch gestures, such as pinches, are typically associ¬ 
ated with a single type of functionality, such as zooming. 
In real life, a pinch gesture could be used for different ac¬ 
tions, e.g., picking up an object or squeezing an object. Our 
study did not find a significant difference in proficiency be¬ 
tween making pinches with the index or middle finger. We 
created a simple mobile photo manipulation application that 
uses Multi^-touch gestures, e.g., multi-touch gestures that af¬ 
ford different types of natural interactions by using different 
fingers in the gesture. A pinch gesture made with the index 
and thumb is used for scaling a photo but a pinch-in gesture 
made with a magnetic fingerprint on the middle finger picks 
up the photo, upon which the user can drag the photo to a 
new location and drop the photo with a pinch-out. Though 
this gesture may be less efficient than using a single finger it 
doesn’t require mode switching and allows for two different 
pinch gestures that both resemble natural interactions. 

Touchscreen Typing 

Virtual keyboards on tablet devices (in landscape mode) 
nearly take up half of the screen; which leaves little screen 
real estate available for applications that rely on typing, such 
as word processors. The spacebar is the most frequently 



Figure 8. By removing the space bar and using a magnetic fingerprint to 
provide spaces, virtual keyboards could be made 25% smaller. 


used key but nearly takes up an entire row. We developed 
a simple text editor application that uses a smaller custom 
keyboard with no spacebar and only three rows of keys us¬ 
ing a minor rearrangement of keys. With a magnetic finger¬ 
print attached to the thumb users can activate the space bar 
by briefly tapping the screen or even using a wireless flick 
gesture. Backspace could be activated using a magnetic fin¬ 
gerprint on the other thumb. 

DISCUSSION AND FUTURE WORK 

Scalability. Our experiments reveal that our technique is lim¬ 
ited to using up to two magnetic fingerprints at the same time 
but magnetic fingerprints cannot be chorded. This limita¬ 
tion is due to magnetic interference, as current magnetome¬ 
ters are unable to resolve the complex magnetic fields that 
appear between using multiple magnets. Despite this limi¬ 
tation magnetic fingerprint significantly increases touch vo¬ 
cabulary. Without chording up to 13 unique (multi) touch 
events can be defined for two magnetic fingerprints and up to 
9 unique (multi) touch events for using one magnetic finger¬ 
print, which nearly doubling (one) or triplesl (two) the current 
touch vocabulary of mobile touchdevices. 

Comparison. We compare our results to a number of closely 
related approaches. A precise comparison is difficult due to 
differences in the increase in touch vocabulary and experi¬ 
mental conditions. Our approach offers a slightly smaller in¬ 
crease in input space as Microrolls [22] (16 micro gestures, 
95% accuracy) but offers a slightly higher accuracy. Thum- 
brock [4] offers a similar accuracy (96%) but only adds a 
single thumb micro gesture. In general, microgestures are a 
slower method of input than our approach [11]. Unlike micro 
gestures, our technique does require a small amount of user 
instrumentation. Tapsense [10] achieves an accuracy of 95% 
and can distinguish four different types of finger touches. 
Gripsense [7] identifies three levels of finger pressure with 
95% accuracy. We achieve an accuracy of 96% for three dif¬ 
ferent types of fingers, but our technique can be used to aug¬ 
ment multitouch input (no chording). Tapsense also requires 
an external acoustic sensor and per user calibration. Other re¬ 
lated approaches [2,5,12] do not report accuracy. Looking at 
other factors, Magnetic fingerprints do not require any exter¬ 
nal sensors or any form of user calibration and it is low-cost 
with neodymium magnets retailing for less than $0.10. 

Limitations. Finger instrumentation is used in a number of 
novel input techniques [6,23,24]. Our approach uses existing 
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sensors and requires the smallest amount of user instrumen¬ 
tation (an untethered small magnet) which seems like a rea¬ 
sonable usability tradeoff to achieve a significant increase in 
input space with a high accuracy. A current limitation of mag¬ 
netic fingerprints is that false recognitions may occur when a 
user does not fully retract their magnetized finger if that finger 
is not involved in the gesture provision. We did not observe 
this interference in our first study for using a single magnetic 
fingerprint, but this problem manifested itself only when us¬ 
ing two and larger sized magnets on the tablet sized touch¬ 
screen, as due to the larger screen users had to reach over 
the magnetometer to provide input. We observed that some 
users already retract those fingers not involved in input espe¬ 
cially when using smartphone based devices, so this seems a 
feasible requirement. Individual classifiers could be used to 
mitigate false recognitions, which may depend on the size of 
the user’s hand or the position of their fingers, but this would 
limit the general applicability of our technique. To keep con¬ 
ditions balanced between trials, our experiments were per¬ 
formed with the device lying fiat on the table. In preliminary 
trials, we did not observe a difference in performance when 
holding the device in the hand. 

Future research. A cube shaped magnet may allows for defin¬ 
ing six unique magnetic fingerprints based on the available 
orientations of the magnet. It may be challenging however 
to fit this magnet on a finger and it may be harder to clas¬ 
sify magnetic fingerprints. To optimize form factor we will 
explore the use of magnetic nail polish though this may not 
generated a strong enough magnetic field. Our study demon¬ 
strated that using larger magnets allows our technique to be 
used on tablets, though false recognitions may increase. To 
circumvent this problem, we will explore bimanual use with 
the hand closest to the magnetometer equipped with a smaller 
magnet and the other hand with larger magnet. 

CONCLUSION 

This paper presents magnetic fingerprints a mobile input tech¬ 
nique that instruments the user’s finger with a small non- 
obtrusive magnet. By incorporating sensing information from 
the device’s magnetometer, touches made with a magnetic 
fingerprint can be distinguished from a non-instrumented fin¬ 
ger. Unlike existing magnetic input technique, magnetic fin¬ 
gerprints take advantage of the rich vocabulary offered by 
multitouch input. Magnetic fingerprints are low-cost, require 
no user calibration and can implemented on current touch¬ 
screen devices. User studies investigate the accuracy, scala¬ 
bility and limitations of this technique, which found that mag¬ 
netic fingerprints cannot be chorded. Useful applications of 
magnetic fingerprints are presented. 
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